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Abstract 


The National Aeronautics and Space Administration is developing advanced technologies to reduce terrorist risk for 
the air transportation system. Decision support tools are needed to help allocate assets to the most promising 
research. An approach to rank ordering technologies (using logic-evolved decision analysis), with risk reduction as 
the metric, is presented. The development of a spanning set of scenarios using a logic-gate tree is described. 
Baseline risk for these scenarios is evaluated with an approximate reasoning model. Illustrative risk and risk- 
reduction results are presented. 


Introduction 


An important element of the national infrastructure is the aviation system. Major efforts are being directed toward 
improving procedures and providing technical measures to reduce the risk of terrorist attacks on the system. Of 
necessity, many of these efforts are short term and are intended to address the current risk in the aftermath of the 
events of September 11, 2001, using existing technology. Improving aviation security also will require the 
introduction of new technology. As part of its Aviation Safety and Security Program (ASSP), the National 
Aeronautics and Space Administration (NASA) is developing technologies for application in this 10- to 20-year 
timeframe. These technologies are directed to all three phases of the aviation system (aircraft, airport, and national 
airspace) and may have the potential to improve security and system capacity. 

By their very nature, many of the NASA technologies under consideration are quite immature. Inevitably, some of 
them will prove to be more suitable than others for a variety of reasons, and there is a strong incentive to direct the 
limited available funding toward the most promising concepts. How can the prioritization of this research be done? 
The first step in answering this question is to decide on a metric to be used to rank order the technologies. Although 
many metrics are available, such as cost and technical risk, the “risk reduction” associated with terrorist attacks was 
chosen as the initial metric for the technology rank ordering. Several steps are required to perform the rank 
ordering: 

• determine a spanning set of terrorist-attack scenarios, 

• develop an inferential model to infer risk for a scenario, 

• evaluate each scenario for baseline risk and the change caused by an introduction of candidate 
technologies, and 

• rank order the technologies using appropriate measures of risk reduction. 

Taken together, these steps represent the essential phases of the decision process that interests NASA in the 
development of a research-portfolio-prioritization scheme for its aviation security initiative. 



The methodology chosen for this problem was logic-evolved decision (LED) analysis (ref. 1), which was developed 
at Los Alamos National Laboratory. LED analysis uses linked, formal logic models to represent the basic functions 
of a decision analysis tool. Each logic model is a directed graph called a logic-gate tree. The process trees are 
developed deductively using general-purpose tree-construction software called led tools (ref. 2). Two process 
trees are needed for the aviation security prioritization analysis — a possibility tree that provides the basis for the 
spanning set of attack scenarios and an inference tree that defines how the risk metric is to be inferred for the 
baseline case and with the assumed introduction of new security technologies. 

LED models are particularly well suited to decision problems that require the integration of expert knowledge that 
resides with a large and diverse set of subject-matter experts. The use of approximate reasoning techniques within 
LED allows accurate reflection of the inferential processes that are used to make judgments when many of the 
factors are highly subjective, as is the case here. Additionally, the results (including uncertainty) are expressed in an 
easily understandable form using this methodology. In this paper, we show how LED analysis is being applied to 
prioritize research for aviation security. We present the logic models that were used to represent the attack scenarios 
and the inferential process for risk, and we describe the interactions with experts that were used to develop these 
models. Also presented are illustrative risk and risk-reduction rankings for candidate technologies associated with 
the security of large, commercial passenger aircraft. 

Possibility Tree for Aircraft- Attack Scenarios 

When terrorist attacks against United States aviation are considered, the overall system conveniently can be divided 
into three target segments: aircraft, airports, and the national airspace. Aircraft, in turn, may be categorized in many 
ways. The division used here is in accordance with the Federal Aviation Administration (FAA) regulations, which 
are based primarily on operational intent and specify particular requirements that are associated with aircraft 
specifications, operations (including security requirements), maintenance, and crew training. In this paper, we 
consider attacks associated with aircraft that operate under Part 121 of the FAA regulations and that carry 
passengers and cargo. Typical aircraft include large passenger aircraft (e.g., Boeing 747s, 767s, 757s, and 737s and 
similar Airbus types), as well as regional jets (e.g., Canadair RJs). All cargo operations (e.g., Federal Express and 
United Parcel Service) with these, as well as smaller aircraft, also are Part 121 operations. However, the size of the 
aircraft is not the critical parameter; identical aircraft will operate under Part 9 1 when they are owned by a business 
or private individual. Business jets (e.g., Boeing Business Jet, Gulfstream IV, and Citation X), along with typical 
general aviation aircraft (e.g., Cessna Cl 72), also operate under Part 91. 

Figure 1 shows the attack- scenario possibility tree for a Part 121 passenger and cargo aircraft (Part 121 PC). The top 
node in the tree is a terrorist attack involving “a Part 121 air operation handling passengers and cargo.” The gate 
associated with this node is an EXCLUSIVE OR (EOR). The inputs to this gate, which appear below and indented to 
the right by convention, are “The attack targets passengers/crew.” (indicated as Node PC in figure 1), “The attack 
targets the aircraft.” (Node AC), and “The attack uses the aircraft as an enabling system.” (Node AES). Each of 
these nodes defines the starting point for a distinct set of attack scenarios to be developed in turn using deduction. 
The gate type for nodes PC and AC is a Continuation (CO), which is used to facilitate the natural-language 
development of the logic. Node AC has a single input “The attack is”, an EOR with the inputs “The attack is on the 
airframe.” (Node AF), and “The attack is on critical on-board systems.” (Node OBS). Each of these nodes is shown 
as an open diamond indicating that further development of these scenarios can be seen by double-clicking in led 
TOOLS. 

Figure 2 shows the continued development of a single airframe attack involving a man-portable missile that 
commonly is referred to as a Man PAD (Node AF-2). Man PAD attacks against a passenger aircraft on departure 
and a successful attack against a cargo aircraft on approach (in Iraq) have been reported. Node AF-2 is defined as a 
PROCESS (PO) gate, and the inputs describe a sequence of steps to be carried out in mounting this attack. Note that in 
a process gate, the order of the inputs from top to bottom indicates the process sequence. The first input to this gate 
describes the type of portable missile used: “ The missile A”. This gate is a taxonomy gate and is useful for 
defining an ordered set of alternatives. In this case, three alternative types of missiles are shown. All three appear as 
solid circles, indicating that they are terminal nodes. Two of these (“a Stinger ” and “ another Man PAD”) appear as 
blue (dark) icons. Blue is a visual indication that a node has been excluded — that is, it will not be considered when 



the possibility tree is evaluated. The next input in the process is “ The attacker acquires the weapon system .” (Node 
AC). 


£5 The targeted system is classified as a Part 1 21 air carrier operation handling passengers and cargo. 
[+] The attack targets passengers/crew. — — - — __ Node PC 

Q The attack 

O involves physical assault 

O involves a chemical agent. 

O involves a dispersible radionuclide. 

O involves a radioactive source. 

0 The attack targets the aircraft. ^ 0 e 

£5 The attack is 

on the airframe, ^ 0 e 

j O The attack originates external to the aircraft. 

■ O The attack originates within the aircraft. 

^ on critical on-board systems. ^ Node OBS 

■■■ O The attack originates external to the aircraft. 

• O The attack originates within the aircraft. 

The attack uses the aircraft as an enabling system, ^ 

■■ O The attack utilises a eomerdal aircraft as a weapon. 

- O The attack uses the aircraft to disperse WMP. 

<> The attack uses the passengers to spread a biological agent. 

O The attack utilizes an aircraft as an explosive weapon after landing. 


Node AES 


Figure 1 - High-Level Development of Part 121 PC Attack Possibility Tree 
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Figure 2 - Sub-Tree for Man PAD Attack on an Aircraft 


This input is also a taxonomy gate. In this case, a box surrounds the icon. This box indicates that the node is a 
replicant, a sub-tree that may be used in multiple places in the tree. Each of the successive inputs describes another 
part of the process. The last input describes the attacker. 




The attack scenario possibility tree is a logical equation written in natural-language form. One class of solutions is 
the set of paths. Each path is a unique attack scenario. For example, the Man PAD scenario is as follows. 


AF-2a: The targeted system is classified as a Part 121 air carrier operation handling passengers and cargo. The 
attack targets the aircraft. The attack is on the airframe. The attack originates external to the aircraft. The attack 
involves weaponry. The weapon used is a man portable missile. The missile is an SA-7. The attacker acquires the 
weapon system. The attack system is purchased. The source is an arms dealer. The attacker transports the missile 
system to the attack site. The attacker acquires the target. The attacker fires the missile. The missile flies to the 
target. The missile warhead detonates near a control surface. The attacker group consists of outsiders only. The 
weapon is located off airport. 

This scenario is unedited and appears exactly as shown from solving the logical equation. The degree of detail in a 
scenario is controlled by the exclusion operation noted above and a companion termination operator. Note that the 
description of the attacker is quite simple: “The attacker group consists of outsiders only.” A companion scenario 
(AF-2b) is identical, except that the attack group involves outsiders and appropriate insiders. In the actual analysis, 
these nodes are expressed in considerably more detail. 

The possibility tree provides a mechanism for describing a very large number of scenarios in compact form. If all of 
the nodes in the possibility tree that are considered here were included in the evaluation, more than 14.2 million 
attack scenarios would exist; many of these attacks are variations of a single basic theme. For evaluating the value 
of security technologies, the goal was to obtain a spanning set of scenarios. For the analysis of Part 121 PC aircraft, 
we used 96 scenarios in the spanning set: 48 basic scenarios multiplied by 2 attacker types. A summary of the 
spanning set is given in table 1 . 


Table 1 - Summary of Attack Scenarios in Spanning Set 


Type of Attack 

Number of 
Scenarios 

Example 

PC: Attack on Passengers and Crew 

8 

Dispersion of chemical agent in passenger 
compartment 

AF : Attack on Airframe 

40 

Missile attack with man-portable system 

OBS: Attack on Critical Onboard Systems 

40 

Jamming or spoofing of navigational aids 

AES: Use of aircraft as an enabling system for 
weapons-of-mass-destruction (WMD) attack 

8 

Variations of 9/1 1 World-Trade-Center attack 


Inference Model for Attack Risk 


As noted earlier, to perform a risk-based prioritization of the research technologies, it is necessary to estimate the 
baseline or current risk. Thus, our starting point is to rank order the spanning set of scenarios. Our inferential model 
for terrorist risk incorporates a game theoretic perspective (ref. 3). The game to be played is asymmetric. Specific 
attackers will choose to attempt only a particular subset of attack scenarios associated with particular targets and to 
employ specific attack modes. They will attempt to allocate their assets to inflict the maximum amount of terror. 
Conversely, the defenders must try to protect all of the targets for which they bear responsibility against all attack 
scenarios. They will attempt to minimize their risk. This model is an example of an extensive game that will be 
played with imperfect knowledge by all players. We cast ourselves as the defender in this game and therefore wish 
to minimize the attack risk. To achieve this goal, we must consider the counter-objective of the attackers — that is, to 
estimate risk, we must make an estimate of the optimum strategy for the attackers. 

Figure 3 shows a segment of the inference tree developed for this analysis. The top of this segment is “Change in 
Risk with New PM Suite”. Here, “PM suite” refers to a set of preventive/mitigative capabilities associated with a 
research technology. The gate type here is an inferential and (IA) used exclusively for inference trees and 
represents the operation of inferring the output node from the inputs. The manner in which this inference actually is 
performed will be discussed in the following paragraphs. As expected, the change in risk is inferred from the current 




risk (Node RC) and the future risk with the PM suite in place (Node RF). The logic used to evaluate these two risk 
variables is the same, and we consider the current risk in more detail here. 
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Figure 3 - Inference Model for Attack Risk 


The risk to the defender associated with a scenario is inferred from the aggregate consequence resulting from the 
attack and the likelihood that the attack is successful. Current risk is inferred from the expected aggregate 
consequences (CA) of a successful attack and the likelihood of a successful attack (LS). The aggregate 
consequences are evaluated based on casualty and economic consequences. In principle, other impacts such as 
psychosocial consequences also could be considered. As with the possibility tree, we use natural-language 
expressions in the deduction of the inferential structure. The objective is to deduce those factors determining risk 
and how they logically combine. The terminal nodes in this model are the input variables that are needed to evaluate 
the risk. A simplified representation for the attacker’s decision process, called the Attacker Model, is shown in 
figure 3, starting with the node “Scenario attempt likelihood with current PM suite.” (Node AM). The basic 
inferential model is a cost/benefit analysis from the attacker’s perspective. It is evident that the attacker’s actual 
decision model is inaccessible to us, and we can only try here to approximate the analysis of a rational opponent. 
The attacker’s cost estimate depends on the inherent difficulty of the attack, as well as the risk of interdiction. 
Similarly, in evaluating benefit, a rational attacker takes into account the perceived gain resulting from a successful 
attack and the likelihood that the attack is in fact successful. This latter node is inferred from the uninhibited success 
likelihood — the chances of success even with no defense — and the attacker’s estimate that the defender can prevent 
the attack. The attacker also must assemble the appropriate agent group to perform the attack. In our simple model, 
we picture some general pool of agents available to the attacker and shrink this pool, depending on the knowledge 
required to carry out the attack. The corresponding defender model (DM) is a simple vulnerability estimate that 
depends again on the uninhibited success likelihood and the likelihood that the attack can be prevented. The 
defenders’ estimate of the chances of an attack succeeding in the absence of any preventive actions may well be 
different from what they believe to be the attackers’ estimate. The prevention likelihood will depend on the 
preventive measures in place and their effectiveness, information that is directly available to the defender. 



Approximate Reasoning Model for Risk 


The inferential model identifies the basic factors and their relationships. At this point in the analysis, a decision 
must be made as to how the risk metric is to be measured using these factors. One approach is to represent the 
inferential process as a series of linked numerical functions. This approach is a natural choice when much of the 
input data are quantitative and when the data relationships easily can be cast in functional form. Consideration of 
the problem at hand suggests that these conditions are not met. Much of the data is qualitative in nature, and the 
nature of the attacker/defender extensive game is primarily one of perception. An analysis methodology that is well 
suited to this set of inferential specifications is approximate reasoning (AR) (ref. 4). AR is intended to emulate the 
type of expert judgment used by subject-matter experts. Variables in AR are linguistic — natural-language 
expressions that can take on a set of discrete natural-language values. A one-to-one correspondence exists between 
the natural language used in the inference tree and these linguistic variables. 

In AR, inferences with linguistic variables are made using formal logical implication defined in a series of linked, 
forward-chaining rule bases. A detailed discussion of AR is beyond the scope of this paper; we offer a short 
example herein to demonstrate the compatibility of AR with the type of expert-based inferential reasoning we wish 
to emulate. From figure 3, we conclude that risk, R , can be inferred from the aggregate consequences, C a , and from 
the likelihood of a successful attack, L s , as 


C a AL s^ R 


( 1 ) 


where => is the implication operator. A linguistic variable takes on the values from its universe of discourse 
(UOD), which is the set of words used to describe it. Here we use R e {Very Low, Low, High, Very High}, C a e 
{Negligible, Moderate, High, Catastrophic}, andZ, e {Very Unlikely, Unlikely, Likely, Nearly Certain}. The rule 
base that implements the logical implication is shown in table 2. The shaded entry in the rule base corresponds to 
the implication “If the Aggregate Consequences are High and the Attack Success Likelihood is Unlikely Then the 
Risk is Low”. Each linguistic variable associated with figure 3 requires a UOD, and a rule base is associated with 
each inferential AND gate in this model. The UODs and the rule bases are chosen to be consonant with the 
judgments of the subject-matter experts for the problem: in this case, intelligence officers, commercial pilots, etc. 


Table 2 - Rule Base for Inferring Risk from Aggregate Consequences and Likelihood of a Successful Attack 


: 
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Catastrophic 

“‘O " CD ~ 

Low 



High 

Very High 

Very High 

High 

Low 

Low 

High 

Very High 

Moderate 

Very Low 

Low 

Low 

High 

Negligible 

Very Low 

Very Low 

Very Low 

Low 


Very Unlikely 

Unlikely 

Likely 

Nearly Certain 

Likelihood of Successful Attack 


To perform evaluations with this AR model, we treat each element in a UOD as a fuzzy set. The ability to express 
ambiguous or hedged expressions of a linguistic variable (e.g., “The aggregate consequences are moderate to high, 
but somewhat closer to moderate”) is encoded using the degree of membership vector, ju(C a ) = [Negligible: 0, 
Moderate: 0.8, High: 0.2, Catastrophic: 0]. The concept of likelihood appears many times in the risk model. 
Likelihood is a natural-language expression associated with outcome uncertainty. We interpret a degree of 
membership for L s (e.g., ju(L s ) = {Very Unlikely: 0.2, Unlikely: 0.4, Likely: 0.3, Nearly Certain: 0] as an expression 
of the possibility that the likelihood is described by each of these descriptors. Note that the sum of these 
memberships is not 1.0 — that is, our expression of outcome uncertainty is imprecise and non-probabilistic. A 



natural-language equivalent might be “the likelihood of a successful attack is in the range from very unlikely to 
likely”. Logical implication with fuzzy/possibilistic inputs to a rule base is performed using the min-max operator 
(ref. 5). For the two vectors given here, the corresponding risk vector is ju(R) = [Very Low: 0.2, Low: 0.4, High: 0.2, 
Nearly Certain: 0], which can be expressed variously as “the risk is low,” “the risk is very low to high,” etc. The 
application of linguistic variables, implication rule bases, and fuzzy sets in AR provides the set of capabilities 
needed to evaluate risk for the aircraft-attack problem. 

Illustrative Attack-Scenario Risk Ranking 

We applied the inference model of figure 3 using the AR implementation discussed previously. This application 
required determining the PM measures currently in place for each scenario and estimating their effectiveness. In 
addition, many of the inferential inputs associated with the attacker model required expert elicitation from 
intelligence experts. The use of this information and the resultant risk ranking of the scenarios cannot be discussed 
here. Instead, we present illustrative results to show how the rank ordering can be used. 

Figure 4 shows illustrative results for the spanning set of scenarios. The results are shown for the attack with and 
without an insider(s) as a member of the attack group. The height of the bars is designed to reflect the memberships 
in the risk UOD. A height of 54 would correspond to a risk with full membership in the “very high” risk set and 
similarly for “high” (18), “low” (6), and “very low” (1.5). Although strictly notional, the figure shows how 
including an insider in the attacker group can increase the risk. This increase occurs because many of the PM 
measures in place for certain scenarios are aimed primarily at preventing intruders or passengers from carrying out 
the attack. For our model scenario, Node AF-2, this is not the case. Including an insider does not increase the 
likelihood of a successful attack. 



Scenario 


Figure 4 - Illustrative Risk Rankings for the Spanning Set of Scenarios 


Technology-Dependent Risk Reduction 


NASA Aircraft Security Technology Portfolio : The objective of the Aviation Security (AvSEC) initiative within 
NASA’s AvSSP is to increase the resiliency of the Air Transport System (ATS) from threats and hostile acts and to 
identify and inform users of potential risks. The AvSEC initiative seeks to reduce the risk to the ATS by limiting 
vulnerabilities, preventing vulnerabilities from being exploited, and mitigating the consequences. The project is 
focused on developing technologies for aircraft and onboard systems. Existing technologies and ongoing efforts 
also will be leveraged to reduce the risk to other components in the ATS, such as the National Airspace System 
(NAS). NASA is considering technologies that will address all aspects of the aviation industry, including 
commercial, business, and general aviation. Technologies will be developed with the recognition that meeting 
security requirements is one consideration in an integrated system that must also meet safety, capacity, and mobility 
requirements. 

Eight specific subprojects within AvSEC deal with PM technologies that address risk reduction for Part 121 PC 
aircraft: lightweight fire- and explosion-resistant (LWFR) materials to reduce the damage to the airframe from an 
attack, protected asset flight systems (PAFS) to prevent intrusion of aircraft into restricted airspace, damage- 
adaptive controls (DAC) to provide automated response to damage to control surfaces and engines from an attack, 
vehicle recovery (VR) to allow for the safe landing of aircraft in association with PAFS intervention, 
electromagmetic emission (EME) protection to prevent interference from external fields with flight-critical systems, 
safe and secure information flow (SASIF) to ensure that voice and data communications (including navigational 
signals) between aircraft and the ground are not compromised, fuel tank inerting (FTI) to limit the damage from 
fires and explosions, and biological and chemical sensing (BIOS) to detect WMD attacks using aircraft. 

Potential Risk Reduction: Each proposed PM suite could be effective against some subset of the spanning set of 
attack scenarios. The actual effectiveness of the PM suites currently is unknown because much of the research 
needed to establish the effectiveness has not yet been begun. For this reason, we decided to treat the effectiveness 
parametrically. The maximum risk reduction occurs when a technology is assumed to be “fully effective” against an 
attack. We then aggregated the inferred risk reduction for a particular PM suite across the spanning set. Figure 5 
shows the preliminary risk reduction determined in this manner for the PM suites. Technologies that are effective 
against many scenarios or a few relatively high-risk scenarios will have the greatest aggregate risk reduction. 



Prevention/Mitigation Suite 

Figure 5 - Aggregate Risk Reduction Potential for Proposed PM Suites 


Discussion and Conclusions 


The work presented here is the initial phase in an ongoing effort to prioritize NASA security technology research. 
Currently, we are working on estimating the current risk for Part 121 all-cargo aircraft and Part 91 and Part 135 
aircraft, as well as for airports and the national air space. Each of these types of operations introduces new attack 
scenarios, as well as significant changes in available PM measures. Similarly, the potential risk reduction associated 
with NASA PM technologies will vary for each set of attack scenarios. In the course of evaluating the risk for these 
various operations, it has become clear that the risk analysis is an iterative process. As more information becomes 
available, the spanning sets may change either in the number of scenarios or in the level of detail used to define a 
particular scenario. In addition, the aviation security system continues to evolve with the introduction of new 
technologies and procedures. These changes also affect risk. For this reason, the risk reduction results presented 
here should also be viewed as preliminary. We are working actively with the NASA technologists to understand 
better how their research reduces risk. 

The use of risk reduction as a single rank-ordering metric should be viewed only as a starting point. Other factors 
clearly play important roles as well. In the near future, cost/benefit analyses will be available for the proposed 
technologies. The results of these analyses will make it possible to reflect in the rankings the value of technologies 
that reduce costs, including the costs attributable to delays caused by existing technologies and procedures. Other 
additions planned for the LED inferential model will consider technical risk and the development of a concept of 
operations for the technologies. 
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